This project focuses on designing and optimizing a convolutional neural network (CNN) for the classification of political memes into conservative and liberal viewpoints, aiming to achieve the highest possible accuracy on a validation set. The process includes employing techniques like early stopping and model checkpointing to monitor performance, and making adjustments to the CNN architecture based on changes in accuracy related to hyperparameters such as filters and layers. The task emphasizes reproducibility through a pre-determined data split and involves the creation of separate data generators for training, validation, and testing. The project’s success will be ultimately evaluated on an independent test set, with learning curves and detailed analysis included in the final report.
For this project, the image dataset was sourced by exploring meme-focused pages on social platforms such as Reddit, Facebook, and Pinterest. A total of 1,000 images were collected, with an equal distribution between conservative and liberal political themes.
Special thanks to Kate Arendes for contributions to the collection process.
import PIL
import numpy as np
from PIL import Image
from keras import layers
from tensorflow import keras
from keras import regularizers
from google.colab import drive
import matplotlib.pyplot as plt
from keras.metrics import Precision
from keras.preprocessing.image import ImageDataGenerator
from sklearn.metrics import accuracy_score, precision_score
First, lets download the images from google drive:# Let's mount the drive to load the images
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Let's set the base directory for loading the political meme images
base_directory = "/content/drive/My Drive/Political Meme Dataset/"
# Let's initialize the ImageDataGenerator with rescaling to normalize pixel values
my_generator = ImageDataGenerator(rescale=1./255)
# Let's set up the training data generator
# This loads images of size 150x150, in batches of 4, with binary class labels
train_generator = my_generator.flow_from_directory(
f"{base_directory}/training/",
target_size=(150, 150),
batch_size=4,
class_mode='binary'
)
# Let's set up the validation data generator
# Loads images of the same size and batch size as the training generator
valid_generator = my_generator.flow_from_directory(
f"{base_directory}/validation/",
target_size=(150, 150),
batch_size=4,
class_mode='binary'
)
# Let's set up the test data generator
# Uses the same parameters for consistency across training, validation, and testing
test_generator = my_generator.flow_from_directory(
f"{base_directory}/test/",
target_size=(150, 150),
batch_size=4,
class_mode='binary'
)
Found 600 images belonging to 2 classes. Found 200 images belonging to 2 classes. Found 200 images belonging to 2 classes.
# Let's load a single image using PIL library.
image = Image.open(f"{base_directory}/training/train_liberal/0f76446d7d65a9e6508a226ae33e8a51--felder-donald-oconnor.jpg")
# Let's get some details about the image.
print("Image Mode -->", image.mode)
print("Image Format --> ", image.format)
print("Image Size -->", image.size)
Image Mode --> RGB Image Format --> JPEG Image Size --> (118, 108)
# Let's display the colored image
plt.imshow(np.asarray(image))
plt.colorbar()
<matplotlib.colorbar.Colorbar at 0x7c726ab276a0>
# Let's convert the input image to grayscale
gs_image = image.convert(mode='L')
# Let's display the grayscale image using matplotlib
plt.imshow(np.asarray(gs_image), cmap='gray')
<matplotlib.image.AxesImage at 0x7c726c344cd0>
# Let's resize the image to 200x200 pixels
img_resized = image.resize((200,200))
# Let's print the size of the resized image to verify the new dimensions
print(img_resized.size)
# Let's display the resized image using matplotlib
plt.imshow(np.asarray(img_resized))
(200, 200)
<matplotlib.image.AxesImage at 0x7c726c1bdf00>
# Let's loop through batches of images from the train generator
for my_batch in train_generator:
images = my_batch[0]
labels = my_batch[1]
# Let's iterate over each image and its corresponding label in the batch
for i in range(len(labels)):
plt.imshow(images[i])
plt.colorbar()
plt.show()
# Let's print the label associated with the image
print(labels[i])
break
1.0
0.0
0.0
0.0
# Let's loop through batches of images from the validation generator
for my_batch in valid_generator:
images = my_batch[0]
labels = my_batch[1]
# Let's iterate over each image and its corresponding label in the batch
for i in range(len(labels)):
plt.imshow(images[i])
plt.colorbar()
plt.show()
# Let's print the label associated with the image
print(labels[i])
break
0.0
1.0
1.0
0.0
# Let's loop through batches of images from the test generator
for my_batch in test_generator:
images = my_batch[0]
labels = my_batch[1]
# Let's iterate over each image and its corresponding label in the batch
for i in range(len(labels)):
plt.imshow(images[i])
plt.colorbar()
plt.show()
# Let's print the label associated with the image
print(labels[i])
break
0.0
0.0
0.0
0.0
# Define the input shape and number of classes
input_shape = (150, 150, 3)
num_classes = 2
# Start defining the model
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Adding a couple more Conv2D and MaxPooling2D layers
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling followed by the classifier
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x) # Common dropout rate for regularization
# Output layer
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize the model
model = keras.Model(inputs=inputs, outputs=outputs)
model.summary()
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 150, 150, 3)] 0
conv2d_17 (Conv2D) (None, 150, 150, 32) 896
max_pooling2d_17 (MaxPooli (None, 75, 75, 32) 0
ng2D)
conv2d_18 (Conv2D) (None, 75, 75, 64) 18496
max_pooling2d_18 (MaxPooli (None, 37, 37, 64) 0
ng2D)
conv2d_19 (Conv2D) (None, 37, 37, 128) 73856
max_pooling2d_19 (MaxPooli (None, 18, 18, 128) 0
ng2D)
conv2d_20 (Conv2D) (None, 18, 18, 128) 147584
max_pooling2d_20 (MaxPooli (None, 9, 9, 128) 0
ng2D)
conv2d_21 (Conv2D) (None, 9, 9, 256) 295168
max_pooling2d_21 (MaxPooli (None, 4, 4, 256) 0
ng2D)
global_average_pooling2d_1 (None, 256) 0
(GlobalAveragePooling2D)
dropout_5 (Dropout) (None, 256) 0
dense_9 (Dense) (None, 1) 257
=================================================================
Total params: 536257 (2.05 MB)
Trainable params: 536257 (2.05 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
filepath="checkpoint_filepath",
save_best_only=True,
monitor="val_loss")
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's train and validate model on the training and validation data
history = model.fit(train_generator, validation_data = valid_generator, epochs = 10, verbose = 1, batch_size = 8, callbacks = [cb_check, cb_early])
Epoch 1/10 150/150 [==============================] - 7s 36ms/step - loss: 0.6890 - accuracy: 0.5767 - precision_1: 0.5821 - val_loss: 0.6557 - val_accuracy: 0.6200 - val_precision_1: 0.5682 Epoch 2/10 150/150 [==============================] - 5s 32ms/step - loss: 0.5482 - accuracy: 0.7567 - precision_1: 0.7188 - val_loss: 0.3334 - val_accuracy: 0.9400 - val_precision_1: 0.8929 Epoch 3/10 150/150 [==============================] - 5s 32ms/step - loss: 0.4263 - accuracy: 0.8567 - precision_1: 0.8242 - val_loss: 0.2132 - val_accuracy: 0.9450 - val_precision_1: 0.9159 Epoch 4/10 150/150 [==============================] - 5s 32ms/step - loss: 0.3202 - accuracy: 0.8750 - precision_1: 0.8462 - val_loss: 0.2123 - val_accuracy: 0.9050 - val_precision_1: 0.9551 Epoch 5/10 150/150 [==============================] - 5s 33ms/step - loss: 0.2947 - accuracy: 0.8983 - precision_1: 0.8842 - val_loss: 0.1506 - val_accuracy: 0.9550 - val_precision_1: 0.9417 Epoch 6/10 150/150 [==============================] - 4s 25ms/step - loss: 0.2917 - accuracy: 0.8917 - precision_1: 0.8660 - val_loss: 0.2518 - val_accuracy: 0.8800 - val_precision_1: 0.8065 Epoch 7/10 150/150 [==============================] - 4s 25ms/step - loss: 0.2616 - accuracy: 0.9067 - precision_1: 0.9013 - val_loss: 0.2803 - val_accuracy: 0.8700 - val_precision_1: 0.7937 Epoch 8/10 150/150 [==============================] - 4s 24ms/step - loss: 0.2586 - accuracy: 0.9033 - precision_1: 0.8854 - val_loss: 0.2528 - val_accuracy: 0.8850 - val_precision_1: 0.8130 Epoch 9/10 150/150 [==============================] - 4s 25ms/step - loss: 0.2721 - accuracy: 0.8867 - precision_1: 0.8718 - val_loss: 0.1694 - val_accuracy: 0.9350 - val_precision_1: 0.8850 Epoch 10/10 150/150 [==============================] - 4s 26ms/step - loss: 0.2068 - accuracy: 0.9167 - precision_1: 0.9058 - val_loss: 0.1603 - val_accuracy: 0.9400 - val_precision_1: 0.8929
train_accuracy = history.history["accuracy"]
train_loss = history.history["loss"]
train_precision = history.history["precision_1"]
val_accuracy = history.history["val_accuracy"]
val_loss = history.history["val_loss"]
val_precision = history.history["val_precision_1"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's loads the best-performing model and evaluate on the test data
model = keras.models.load_model("checkpoint_filepath")
model.evaluate(test_generator)
50/50 [==============================] - 73s 1s/step - loss: 0.2016 - accuracy: 0.9500 - precision_1: 0.9245
[0.20159845054149628, 0.949999988079071, 0.9245283007621765]
The initial model architecture resulted in a training accuracy of 0.9167 and a validation accuracy of 0.94 after being trained for 10 epochs. The next steps include increasing the number of epochs to 50 to observe how the training and validation accuracies change across different epochs. This extended training period will help determine if the model is benefiting from more training time or if it begins to overfit the training data. Observing the trend in validation accuracy will also indicate whether the model generalizes well to unseen data. Additional measures, such as implementing early stopping or adjusting the learning rate, may be considered based on the outcomes observed at different epochs.
# Define the input shape and number of classes
input_shape = (150, 150, 3)
num_classes = 2
# Start defining the model
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Adding a couple more Conv2D and MaxPooling2D layers
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling followed by the classifier
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x) # Common dropout rate for regularization
# Output layer
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize the model
model_increase_epochs = keras.Model(inputs=inputs, outputs=outputs)
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model_increase_epochs.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
filepath="base_model_checkpoint_filepath",
save_best_only=True,
monitor="val_loss")
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's train and validate model on the training and validation data
history_increase_epochs = model_increase_epochs.fit(train_generator, validation_data = valid_generator, epochs = 30, verbose = 1, batch_size = 8, callbacks = [cb_check, cb_early])
Epoch 1/30 150/150 [==============================] - 7s 34ms/step - loss: 0.6025 - accuracy: 0.6800 - precision_2: 0.7673 - val_loss: 0.2778 - val_accuracy: 0.9400 - val_precision_2: 0.9314 Epoch 2/30 150/150 [==============================] - 4s 25ms/step - loss: 0.4617 - accuracy: 0.8133 - precision_2: 0.7749 - val_loss: 0.2888 - val_accuracy: 0.9550 - val_precision_2: 0.9333 Epoch 3/30 150/150 [==============================] - 5s 32ms/step - loss: 0.4052 - accuracy: 0.8600 - precision_2: 0.8396 - val_loss: 0.1892 - val_accuracy: 0.9550 - val_precision_2: 0.9417 Epoch 4/30 150/150 [==============================] - 5s 34ms/step - loss: 0.3427 - accuracy: 0.8683 - precision_2: 0.8421 - val_loss: 0.1826 - val_accuracy: 0.9400 - val_precision_2: 0.8929 Epoch 5/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2917 - accuracy: 0.8833 - precision_2: 0.8571 - val_loss: 0.3593 - val_accuracy: 0.8400 - val_precision_2: 0.9722 Epoch 6/30 150/150 [==============================] - 4s 26ms/step - loss: 0.3290 - accuracy: 0.8750 - precision_2: 0.8571 - val_loss: 0.2781 - val_accuracy: 0.8900 - val_precision_2: 0.8197 Epoch 7/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2959 - accuracy: 0.8983 - precision_2: 0.8794 - val_loss: 0.1890 - val_accuracy: 0.9350 - val_precision_2: 0.9394 Epoch 8/30 150/150 [==============================] - 5s 32ms/step - loss: 0.3395 - accuracy: 0.8550 - precision_2: 0.8257 - val_loss: 0.1690 - val_accuracy: 0.9700 - val_precision_2: 0.9519 Epoch 9/30 150/150 [==============================] - 5s 32ms/step - loss: 0.2545 - accuracy: 0.9167 - precision_2: 0.8956 - val_loss: 0.1290 - val_accuracy: 0.9550 - val_precision_2: 0.9174 Epoch 10/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2300 - accuracy: 0.9133 - precision_2: 0.9000 - val_loss: 0.1414 - val_accuracy: 0.9400 - val_precision_2: 0.8929 Epoch 11/30 150/150 [==============================] - 5s 33ms/step - loss: 0.2220 - accuracy: 0.9167 - precision_2: 0.9058 - val_loss: 0.1166 - val_accuracy: 0.9700 - val_precision_2: 0.9519 Epoch 12/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1804 - accuracy: 0.9317 - precision_2: 0.9191 - val_loss: 0.2790 - val_accuracy: 0.8800 - val_precision_2: 0.8065 Epoch 13/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1969 - accuracy: 0.9233 - precision_2: 0.9097 - val_loss: 0.1492 - val_accuracy: 0.9500 - val_precision_2: 0.9091 Epoch 14/30 150/150 [==============================] - 5s 33ms/step - loss: 0.1829 - accuracy: 0.9300 - precision_2: 0.9188 - val_loss: 0.1068 - val_accuracy: 0.9650 - val_precision_2: 0.9429 Epoch 15/30 150/150 [==============================] - 5s 34ms/step - loss: 0.1340 - accuracy: 0.9567 - precision_2: 0.9536 - val_loss: 0.1021 - val_accuracy: 0.9600 - val_precision_2: 0.9340 Epoch 16/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1566 - accuracy: 0.9467 - precision_2: 0.9408 - val_loss: 0.3301 - val_accuracy: 0.8950 - val_precision_2: 0.8264 Epoch 17/30 150/150 [==============================] - 4s 27ms/step - loss: 0.1869 - accuracy: 0.9333 - precision_2: 0.9248 - val_loss: 0.1835 - val_accuracy: 0.9350 - val_precision_2: 0.8850 Epoch 18/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1433 - accuracy: 0.9483 - precision_2: 0.9529 - val_loss: 0.1376 - val_accuracy: 0.9400 - val_precision_2: 0.8929 Epoch 19/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1453 - accuracy: 0.9467 - precision_2: 0.9379 - val_loss: 0.1894 - val_accuracy: 0.9350 - val_precision_2: 0.8850 Epoch 20/30 150/150 [==============================] - 5s 36ms/step - loss: 0.1103 - accuracy: 0.9517 - precision_2: 0.9593 - val_loss: 0.0948 - val_accuracy: 0.9750 - val_precision_2: 0.9798 Epoch 21/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0834 - accuracy: 0.9667 - precision_2: 0.9667 - val_loss: 0.2334 - val_accuracy: 0.9250 - val_precision_2: 0.8696 Epoch 22/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0864 - accuracy: 0.9650 - precision_2: 0.9604 - val_loss: 0.1216 - val_accuracy: 0.9500 - val_precision_2: 0.9245 Epoch 23/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0735 - accuracy: 0.9750 - precision_2: 0.9703 - val_loss: 0.1159 - val_accuracy: 0.9650 - val_precision_2: 0.9346 Epoch 24/30 150/150 [==============================] - 5s 33ms/step - loss: 0.0852 - accuracy: 0.9617 - precision_2: 0.9632 - val_loss: 0.0707 - val_accuracy: 0.9700 - val_precision_2: 0.9434 Epoch 25/30 150/150 [==============================] - 5s 33ms/step - loss: 0.0523 - accuracy: 0.9833 - precision_2: 0.9801 - val_loss: 0.0654 - val_accuracy: 0.9800 - val_precision_2: 0.9800 Epoch 26/30 150/150 [==============================] - 4s 27ms/step - loss: 0.0563 - accuracy: 0.9783 - precision_2: 0.9767 - val_loss: 0.1835 - val_accuracy: 0.9500 - val_precision_2: 0.9091 Epoch 27/30 150/150 [==============================] - 4s 24ms/step - loss: 0.0925 - accuracy: 0.9617 - precision_2: 0.9601 - val_loss: 0.1404 - val_accuracy: 0.9500 - val_precision_2: 0.9167 Epoch 28/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0781 - accuracy: 0.9733 - precision_2: 0.9702 - val_loss: 0.0910 - val_accuracy: 0.9700 - val_precision_2: 0.9434 Epoch 29/30 150/150 [==============================] - 4s 27ms/step - loss: 0.0469 - accuracy: 0.9817 - precision_2: 0.9801 - val_loss: 0.1405 - val_accuracy: 0.9550 - val_precision_2: 0.9252 Epoch 30/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0180 - accuracy: 0.9983 - precision_2: 0.9967 - val_loss: 0.1724 - val_accuracy: 0.9650 - val_precision_2: 0.9429
train_accuracy = history_increase_epochs.history["accuracy"]
train_loss = history_increase_epochs.history["loss"]
train_precision = history_increase_epochs.history["precision_2"]
val_accuracy = history_increase_epochs.history["val_accuracy"]
val_loss = history_increase_epochs.history["val_loss"]
val_precision = history_increase_epochs.history["val_precision_2"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's loads the best-performing model and evaluate on the test data
best_model = keras.models.load_model("base_model_checkpoint_filepath")
best_model.evaluate(test_generator)
50/50 [==============================] - 1s 16ms/step - loss: 0.1484 - accuracy: 0.9550 - precision_2: 0.9333
[0.14840058982372284, 0.9549999833106995, 0.9333333373069763]
Increasing the number of epochs from 10 to 30 resulted in an increase in training accuracy from 0.9169 to 0.9983. Similarly, the validation accuracy increased from 0.94 to 0.97. As expected, the time required for training increased significantly from 3 minutes to 5 minutes due to the higher number of epochs. This indicates that the model benefited from additional training, as evidenced by the improvements in both training and validation accuracies.
# Define the input shape and number of classes
input_shape = (150, 150, 3)
num_classes = 2
# Start defining the model
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Adding a couple more Conv2D and MaxPooling2D layers
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling followed by the classifier
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x) # Common dropout rate for regularization
# Output layer
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize the model
model_decrease_layers = keras.Model(inputs=inputs, outputs=outputs)
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model_decrease_layers.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
filepath="decrease_layers_checkpoint_filepath",
save_best_only=True,
monitor="val_loss")
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's train and validate model on the training and validation data
history_decrease_layers = model_decrease_layers.fit(train_generator, validation_data = valid_generator, epochs = 30, verbose = 1, batch_size = 8, callbacks = [cb_check, cb_early])
Epoch 1/30 150/150 [==============================] - 7s 36ms/step - loss: 0.6384 - accuracy: 0.6117 - precision_6: 0.6683 - val_loss: 0.3889 - val_accuracy: 0.8500 - val_precision_6: 0.9268 Epoch 2/30 150/150 [==============================] - 5s 31ms/step - loss: 0.5383 - accuracy: 0.8133 - precision_6: 0.7848 - val_loss: 0.2210 - val_accuracy: 0.9300 - val_precision_6: 0.9216 Epoch 3/30 150/150 [==============================] - 4s 25ms/step - loss: 0.4117 - accuracy: 0.8483 - precision_6: 0.8047 - val_loss: 0.3755 - val_accuracy: 0.8250 - val_precision_6: 0.7407 Epoch 4/30 150/150 [==============================] - 5s 31ms/step - loss: 0.3692 - accuracy: 0.8600 - precision_6: 0.8214 - val_loss: 0.1845 - val_accuracy: 0.9300 - val_precision_6: 0.9574 Epoch 5/30 150/150 [==============================] - 4s 26ms/step - loss: 0.3661 - accuracy: 0.8850 - precision_6: 0.8468 - val_loss: 0.1889 - val_accuracy: 0.9600 - val_precision_6: 0.9423 Epoch 6/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3195 - accuracy: 0.8883 - precision_6: 0.8585 - val_loss: 0.2010 - val_accuracy: 0.9350 - val_precision_6: 0.8850 Epoch 7/30 150/150 [==============================] - 5s 31ms/step - loss: 0.3544 - accuracy: 0.8900 - precision_6: 0.8503 - val_loss: 0.1639 - val_accuracy: 0.9700 - val_precision_6: 0.9608 Epoch 8/30 150/150 [==============================] - 4s 26ms/step - loss: 0.3111 - accuracy: 0.8867 - precision_6: 0.8558 - val_loss: 0.3436 - val_accuracy: 0.8400 - val_precision_6: 0.7576 Epoch 9/30 150/150 [==============================] - 5s 31ms/step - loss: 0.2674 - accuracy: 0.9050 - precision_6: 0.8932 - val_loss: 0.1489 - val_accuracy: 0.9450 - val_precision_6: 0.9495 Epoch 10/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2787 - accuracy: 0.8900 - precision_6: 0.8634 - val_loss: 0.1836 - val_accuracy: 0.9600 - val_precision_6: 0.9340 Epoch 11/30 150/150 [==============================] - 5s 31ms/step - loss: 0.2507 - accuracy: 0.9100 - precision_6: 0.8968 - val_loss: 0.1213 - val_accuracy: 0.9700 - val_precision_6: 0.9519 Epoch 12/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2745 - accuracy: 0.8967 - precision_6: 0.8889 - val_loss: 0.2140 - val_accuracy: 0.9050 - val_precision_6: 0.8403 Epoch 13/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2572 - accuracy: 0.9083 - precision_6: 0.8889 - val_loss: 0.1525 - val_accuracy: 0.9600 - val_precision_6: 0.9259 Epoch 14/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2253 - accuracy: 0.9183 - precision_6: 0.9088 - val_loss: 0.1747 - val_accuracy: 0.9350 - val_precision_6: 0.8850 Epoch 15/30 150/150 [==============================] - 5s 31ms/step - loss: 0.2156 - accuracy: 0.9133 - precision_6: 0.8949 - val_loss: 0.1048 - val_accuracy: 0.9650 - val_precision_6: 0.9515 Epoch 16/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2030 - accuracy: 0.9300 - precision_6: 0.9135 - val_loss: 0.1429 - val_accuracy: 0.9400 - val_precision_6: 0.8929 Epoch 17/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1842 - accuracy: 0.9283 - precision_6: 0.9186 - val_loss: 0.1763 - val_accuracy: 0.9100 - val_precision_6: 0.8475 Epoch 18/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2090 - accuracy: 0.9183 - precision_6: 0.8959 - val_loss: 0.2044 - val_accuracy: 0.9000 - val_precision_6: 0.8333 Epoch 19/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1977 - accuracy: 0.9233 - precision_6: 0.9097 - val_loss: 0.1396 - val_accuracy: 0.9550 - val_precision_6: 0.9174 Epoch 20/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2096 - accuracy: 0.9217 - precision_6: 0.9148 - val_loss: 0.1372 - val_accuracy: 0.9700 - val_precision_6: 0.9434 Epoch 21/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1763 - accuracy: 0.9383 - precision_6: 0.9283 - val_loss: 0.1483 - val_accuracy: 0.9450 - val_precision_6: 0.9009 Epoch 22/30 150/150 [==============================] - 5s 31ms/step - loss: 0.1603 - accuracy: 0.9383 - precision_6: 0.9311 - val_loss: 0.0925 - val_accuracy: 0.9700 - val_precision_6: 0.9434 Epoch 23/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1325 - accuracy: 0.9533 - precision_6: 0.9444 - val_loss: 0.1019 - val_accuracy: 0.9650 - val_precision_6: 0.9346 Epoch 24/30 150/150 [==============================] - 5s 34ms/step - loss: 0.1211 - accuracy: 0.9533 - precision_6: 0.9474 - val_loss: 0.0873 - val_accuracy: 0.9700 - val_precision_6: 0.9608 Epoch 25/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1849 - accuracy: 0.9333 - precision_6: 0.9167 - val_loss: 0.1551 - val_accuracy: 0.9300 - val_precision_6: 0.8772 Epoch 26/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1315 - accuracy: 0.9467 - precision_6: 0.9408 - val_loss: 0.1343 - val_accuracy: 0.9450 - val_precision_6: 0.9238 Epoch 27/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1170 - accuracy: 0.9617 - precision_6: 0.9601 - val_loss: 0.1024 - val_accuracy: 0.9650 - val_precision_6: 0.9346 Epoch 28/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0982 - accuracy: 0.9650 - precision_6: 0.9574 - val_loss: 0.2915 - val_accuracy: 0.8850 - val_precision_6: 0.8130 Epoch 29/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0881 - accuracy: 0.9667 - precision_6: 0.9605 - val_loss: 0.1394 - val_accuracy: 0.9400 - val_precision_6: 0.8929 Epoch 30/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0770 - accuracy: 0.9683 - precision_6: 0.9637 - val_loss: 0.1180 - val_accuracy: 0.9500 - val_precision_6: 0.9500
train_accuracy = history_decrease_layers.history["accuracy"]
train_loss = history_decrease_layers.history["loss"]
train_precision = history_decrease_layers.history["precision_6"]
val_accuracy = history_decrease_layers.history["val_accuracy"]
val_loss = history_decrease_layers.history["val_loss"]
val_precision = history_decrease_layers.history["val_precision_6"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's loads the best-performing model and evaluate on the test data
model = keras.models.load_model("decrease_layers_checkpoint_filepath")
model.evaluate(test_generator)
50/50 [==============================] - 1s 15ms/step - loss: 0.1241 - accuracy: 0.9400 - precision_6: 0.9000
[0.12408556789159775, 0.9399999976158142, 0.8999999761581421]
Surprisingly, decreasing one convolution and pooling layers by resulted in a training accuracy of 0.9683 and a validation accuracy of 0.95. The time it took to complete training and validation for 30 epochs was almost 3 minutes. The test performance is similar to the configuration with more layers; the accuracy with fewer layers is 0.94, while the accuracy with more layers is 0.9550.
# Define the input shape and number of classes
input_shape = (150, 150, 3)
num_classes = 2
# Start defining the model
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Adding a couple more Conv2D and MaxPooling2D layers
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling followed by the classifier
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x) # Common dropout rate for regularization
# Output layer
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize the model
model_increased_filters = keras.Model(inputs=inputs, outputs=outputs)
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model_increased_filters.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
filepath="increase_filters_checkpoint_filepath",
save_best_only=True,
monitor="val_loss")
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's train and validate model on the training and validation data
history_increased_filters = model_increased_filters.fit(train_generator, validation_data = valid_generator, epochs = 30, verbose = 1, batch_size = 8, callbacks = [cb_check, cb_early])
Epoch 1/30 150/150 [==============================] - 7s 35ms/step - loss: 0.6311 - accuracy: 0.6733 - precision_9: 0.6711 - val_loss: 0.3597 - val_accuracy: 0.9000 - val_precision_9: 0.8448 Epoch 2/30 150/150 [==============================] - 4s 25ms/step - loss: 0.5611 - accuracy: 0.7550 - precision_9: 0.7029 - val_loss: 0.6817 - val_accuracy: 0.5150 - val_precision_9: 0.5076 Epoch 3/30 150/150 [==============================] - 4s 26ms/step - loss: 0.6183 - accuracy: 0.7517 - precision_9: 0.6921 - val_loss: 0.4809 - val_accuracy: 0.8200 - val_precision_9: 0.7353 Epoch 4/30 150/150 [==============================] - 5s 35ms/step - loss: 0.5312 - accuracy: 0.7850 - precision_9: 0.7221 - val_loss: 0.2999 - val_accuracy: 0.9100 - val_precision_9: 0.9457 Epoch 5/30 150/150 [==============================] - 5s 33ms/step - loss: 0.4459 - accuracy: 0.8367 - precision_9: 0.7953 - val_loss: 0.2461 - val_accuracy: 0.9050 - val_precision_9: 0.8403 Epoch 6/30 150/150 [==============================] - 5s 33ms/step - loss: 0.3995 - accuracy: 0.8500 - precision_9: 0.8125 - val_loss: 0.1975 - val_accuracy: 0.9450 - val_precision_9: 0.9406 Epoch 7/30 150/150 [==============================] - 5s 33ms/step - loss: 0.3167 - accuracy: 0.8800 - precision_9: 0.8608 - val_loss: 0.1454 - val_accuracy: 0.9600 - val_precision_9: 0.9259 Epoch 8/30 150/150 [==============================] - 5s 34ms/step - loss: 0.2897 - accuracy: 0.8883 - precision_9: 0.8698 - val_loss: 0.1401 - val_accuracy: 0.9550 - val_precision_9: 0.9174 Epoch 9/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2840 - accuracy: 0.9033 - precision_9: 0.8878 - val_loss: 0.2197 - val_accuracy: 0.9100 - val_precision_9: 0.8475 Epoch 10/30 150/150 [==============================] - 4s 26ms/step - loss: 0.3070 - accuracy: 0.8967 - precision_9: 0.8719 - val_loss: 0.1498 - val_accuracy: 0.9600 - val_precision_9: 0.9340 Epoch 11/30 150/150 [==============================] - 5s 33ms/step - loss: 0.2452 - accuracy: 0.9083 - precision_9: 0.8990 - val_loss: 0.0774 - val_accuracy: 0.9850 - val_precision_9: 0.9709 Epoch 12/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2598 - accuracy: 0.9200 - precision_9: 0.9145 - val_loss: 0.1180 - val_accuracy: 0.9800 - val_precision_9: 0.9615 Epoch 13/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2308 - accuracy: 0.9200 - precision_9: 0.9013 - val_loss: 0.1339 - val_accuracy: 0.9550 - val_precision_9: 0.9174 Epoch 14/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2041 - accuracy: 0.9233 - precision_9: 0.9123 - val_loss: 0.1062 - val_accuracy: 0.9700 - val_precision_9: 0.9434 Epoch 15/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1899 - accuracy: 0.9300 - precision_9: 0.9216 - val_loss: 0.0901 - val_accuracy: 0.9750 - val_precision_9: 0.9524 Epoch 16/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1939 - accuracy: 0.9317 - precision_9: 0.9274 - val_loss: 0.0812 - val_accuracy: 0.9800 - val_precision_9: 0.9706 Epoch 17/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1993 - accuracy: 0.9317 - precision_9: 0.9302 - val_loss: 0.0855 - val_accuracy: 0.9700 - val_precision_9: 0.9434 Epoch 18/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1534 - accuracy: 0.9467 - precision_9: 0.9379 - val_loss: 0.1325 - val_accuracy: 0.9600 - val_precision_9: 0.9259 Epoch 19/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1672 - accuracy: 0.9383 - precision_9: 0.9340 - val_loss: 0.2987 - val_accuracy: 0.9100 - val_precision_9: 0.8475 Epoch 20/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1494 - accuracy: 0.9433 - precision_9: 0.9346 - val_loss: 0.1611 - val_accuracy: 0.9350 - val_precision_9: 0.8919 Epoch 21/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1508 - accuracy: 0.9517 - precision_9: 0.9502 - val_loss: 0.2378 - val_accuracy: 0.8950 - val_precision_9: 0.8264 Epoch 22/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1223 - accuracy: 0.9500 - precision_9: 0.9470 - val_loss: 0.1070 - val_accuracy: 0.9600 - val_precision_9: 0.9340 Epoch 23/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1350 - accuracy: 0.9533 - precision_9: 0.9474 - val_loss: 0.1106 - val_accuracy: 0.9600 - val_precision_9: 0.9259 Epoch 24/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1030 - accuracy: 0.9650 - precision_9: 0.9604 - val_loss: 0.0970 - val_accuracy: 0.9750 - val_precision_9: 0.9524 Epoch 25/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0725 - accuracy: 0.9767 - precision_9: 0.9735 - val_loss: 0.2406 - val_accuracy: 0.9100 - val_precision_9: 0.8475 Epoch 26/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1095 - accuracy: 0.9667 - precision_9: 0.9667 - val_loss: 0.2239 - val_accuracy: 0.8900 - val_precision_9: 0.8197 Epoch 27/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0797 - accuracy: 0.9767 - precision_9: 0.9735 - val_loss: 0.1151 - val_accuracy: 0.9400 - val_precision_9: 0.8929 Epoch 28/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0820 - accuracy: 0.9750 - precision_9: 0.9642 - val_loss: 0.1602 - val_accuracy: 0.9500 - val_precision_9: 0.9091 Epoch 29/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1663 - accuracy: 0.9467 - precision_9: 0.9379 - val_loss: 0.0975 - val_accuracy: 0.9650 - val_precision_9: 0.9346 Epoch 30/30 150/150 [==============================] - 5s 33ms/step - loss: 0.0960 - accuracy: 0.9667 - precision_9: 0.9575 - val_loss: 0.0504 - val_accuracy: 0.9750 - val_precision_9: 0.9524
train_accuracy = history_increased_filters.history["accuracy"]
train_loss = history_increased_filters.history["loss"]
train_precision = history_increased_filters.history["precision_9"]
val_accuracy = history_increased_filters.history["val_accuracy"]
val_loss = history_increased_filters.history["val_loss"]
val_precision = history_increased_filters.history["val_precision_9"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's loads the best-performing model and evaluate on the test data
model = keras.models.load_model("increase_filters_checkpoint_filepath")
model.evaluate(test_generator)
50/50 [==============================] - 1s 17ms/step - loss: 0.1671 - accuracy: 0.9450 - precision_9: 0.9009
[0.1671462506055832, 0.9449999928474426, 0.9009009003639221]
Surprisingly, increasing the number of convolution filters from 128 to 256 in two convolution layers resulted in a training accuracy of 0.9667 and a validation accuracy of 0.9750. The time it took to complete training and validation for 30 epochs was close to 3 minutes. The test performance is same to the configuration with more filters; the accuracy with fewer filters is 0.9450, while the accuracy with base configuration is 0.9550.
# Define the input shape and number of classes
input_shape = (100, 100, 3)
num_classes = 2
# Start defining the model
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Adding a couple more Conv2D and MaxPooling2D layers
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling followed by the classifier
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x) # Common dropout rate for regularization
# Output layer
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize the model
model_decreased_image_size = keras.Model(inputs=inputs, outputs=outputs)
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model_decreased_image_size.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
filepath="decrease_image_checkpoint_filepath",
save_best_only=True,
monitor="val_loss")
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's train and validate model on the training and validation data
history_decreased_image_size = model_decreased_image_size.fit(train_generator, validation_data = valid_generator, epochs = 30, verbose = 1, batch_size = 8, callbacks = [cb_check, cb_early])
Epoch 1/30 150/150 [==============================] - 7s 35ms/step - loss: 0.6771 - accuracy: 0.5433 - precision_10: 0.5684 - val_loss: 0.4984 - val_accuracy: 0.8500 - val_precision_10: 0.9487 Epoch 2/30 150/150 [==============================] - 4s 25ms/step - loss: 0.6645 - accuracy: 0.6917 - precision_10: 0.6448 - val_loss: 0.6203 - val_accuracy: 0.8000 - val_precision_10: 0.8947 Epoch 3/30 150/150 [==============================] - 5s 33ms/step - loss: 0.5489 - accuracy: 0.7933 - precision_10: 0.7588 - val_loss: 0.2703 - val_accuracy: 0.9100 - val_precision_10: 0.8475 Epoch 4/30 150/150 [==============================] - 4s 25ms/step - loss: 0.4003 - accuracy: 0.8517 - precision_10: 0.8187 - val_loss: 0.3547 - val_accuracy: 0.8450 - val_precision_10: 0.7674 Epoch 5/30 150/150 [==============================] - 5s 33ms/step - loss: 0.3474 - accuracy: 0.8750 - precision_10: 0.8483 - val_loss: 0.2459 - val_accuracy: 0.9100 - val_precision_10: 0.8534 Epoch 6/30 150/150 [==============================] - 5s 35ms/step - loss: 0.3309 - accuracy: 0.8717 - precision_10: 0.8431 - val_loss: 0.1803 - val_accuracy: 0.9400 - val_precision_10: 0.9400 Epoch 7/30 150/150 [==============================] - 5s 33ms/step - loss: 0.2782 - accuracy: 0.8967 - precision_10: 0.8719 - val_loss: 0.1684 - val_accuracy: 0.9350 - val_precision_10: 0.9780 Epoch 8/30 150/150 [==============================] - 4s 26ms/step - loss: 0.3530 - accuracy: 0.8683 - precision_10: 0.8421 - val_loss: 0.1783 - val_accuracy: 0.9550 - val_precision_10: 0.9174 Epoch 9/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2585 - accuracy: 0.9150 - precision_10: 0.9055 - val_loss: 0.1725 - val_accuracy: 0.9350 - val_precision_10: 0.8850 Epoch 10/30 150/150 [==============================] - 5s 33ms/step - loss: 0.2449 - accuracy: 0.9083 - precision_10: 0.8964 - val_loss: 0.1183 - val_accuracy: 0.9750 - val_precision_10: 0.9524 Epoch 11/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2045 - accuracy: 0.9217 - precision_10: 0.9175 - val_loss: 0.2540 - val_accuracy: 0.8950 - val_precision_10: 0.8264 Epoch 12/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2040 - accuracy: 0.9250 - precision_10: 0.9264 - val_loss: 0.2036 - val_accuracy: 0.9100 - val_precision_10: 0.8475 Epoch 13/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2061 - accuracy: 0.9217 - precision_10: 0.9121 - val_loss: 0.1259 - val_accuracy: 0.9550 - val_precision_10: 0.9174 Epoch 14/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1878 - accuracy: 0.9333 - precision_10: 0.9392 - val_loss: 0.1239 - val_accuracy: 0.9650 - val_precision_10: 0.9346 Epoch 15/30 150/150 [==============================] - 5s 32ms/step - loss: 0.1955 - accuracy: 0.9250 - precision_10: 0.9293 - val_loss: 0.0922 - val_accuracy: 0.9700 - val_precision_10: 0.9434 Epoch 16/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1650 - accuracy: 0.9450 - precision_10: 0.9406 - val_loss: 0.1260 - val_accuracy: 0.9650 - val_precision_10: 0.9515 Epoch 17/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1498 - accuracy: 0.9467 - precision_10: 0.9467 - val_loss: 0.1421 - val_accuracy: 0.9750 - val_precision_10: 0.9612 Epoch 18/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1711 - accuracy: 0.9400 - precision_10: 0.9314 - val_loss: 0.1141 - val_accuracy: 0.9600 - val_precision_10: 0.9259 Epoch 19/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1386 - accuracy: 0.9533 - precision_10: 0.9533 - val_loss: 0.1124 - val_accuracy: 0.9700 - val_precision_10: 0.9519 Epoch 20/30 150/150 [==============================] - 5s 33ms/step - loss: 0.1379 - accuracy: 0.9483 - precision_10: 0.9410 - val_loss: 0.0620 - val_accuracy: 0.9750 - val_precision_10: 0.9524 Epoch 21/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1545 - accuracy: 0.9433 - precision_10: 0.9375 - val_loss: 0.2216 - val_accuracy: 0.9100 - val_precision_10: 0.8475 Epoch 22/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1055 - accuracy: 0.9550 - precision_10: 0.9565 - val_loss: 0.0851 - val_accuracy: 0.9650 - val_precision_10: 0.9429 Epoch 23/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1115 - accuracy: 0.9600 - precision_10: 0.9600 - val_loss: 0.0859 - val_accuracy: 0.9650 - val_precision_10: 0.9346 Epoch 24/30 150/150 [==============================] - 4s 24ms/step - loss: 0.0806 - accuracy: 0.9733 - precision_10: 0.9671 - val_loss: 0.1564 - val_accuracy: 0.9600 - val_precision_10: 0.9259 Epoch 25/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0836 - accuracy: 0.9700 - precision_10: 0.9669 - val_loss: 0.1628 - val_accuracy: 0.9550 - val_precision_10: 0.9174 Epoch 26/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0939 - accuracy: 0.9700 - precision_10: 0.9638 - val_loss: 0.0989 - val_accuracy: 0.9750 - val_precision_10: 0.9524 Epoch 27/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0828 - accuracy: 0.9700 - precision_10: 0.9608 - val_loss: 0.1323 - val_accuracy: 0.9600 - val_precision_10: 0.9259 Epoch 28/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1188 - accuracy: 0.9583 - precision_10: 0.9421 - val_loss: 0.1243 - val_accuracy: 0.9400 - val_precision_10: 0.8929 Epoch 29/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1644 - accuracy: 0.9500 - precision_10: 0.9441 - val_loss: 0.0734 - val_accuracy: 0.9800 - val_precision_10: 0.9615 Epoch 30/30 150/150 [==============================] - 5s 33ms/step - loss: 0.1053 - accuracy: 0.9650 - precision_10: 0.9666 - val_loss: 0.0532 - val_accuracy: 0.9900 - val_precision_10: 0.9804
train_accuracy = history_decreased_image_size.history["accuracy"]
train_loss = history_decreased_image_size.history["loss"]
train_precision = history_decreased_image_size.history["precision_10"]
val_accuracy = history_decreased_image_size.history["val_accuracy"]
val_loss = history_decreased_image_size.history["val_loss"]
val_precision = history_decreased_image_size.history["val_precision_10"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's loads the best-performing model and evaluate on the test data
model = keras.models.load_model("decrease_image_checkpoint_filepath")
model.evaluate(test_generator)
50/50 [==============================] - 1s 16ms/step - loss: 0.1189 - accuracy: 0.9600 - precision_10: 0.9423
[0.11893752962350845, 0.9599999785423279, 0.942307710647583]
Decreasing the size of input images by almost 40% resulted in a training accuracy of 0.9650 and a validation accuracy of 0.99. The time it took to complete training and validation for 30 epochs was close to 3 minutes. The test performance is almost identical to the configuration with larger input sizes; the accuracy with the smaller image size is 0.96, while the accuracy with the larger image size is 0.9550.
# Let's load the best performing model
best_model = keras.models.load_model("decrease_image_checkpoint_filepath")
import matplotlib.image as mpimg
# Read the image
img = mpimg.imread("image.png")
# Create a new figure with a high DPI
plt.figure(dpi=300) # Increase the value for higher quality images
# Display the image
plt.imshow(img)
plt.axis("off") # Turn off the axis
# Show the plot with a high-quality image
plt.show()
# Let's set up the validation data generator
# Loads images of the same size and batch size as the training generator
valid_generator = my_generator.flow_from_directory(
f"{base_directory}/validation/",
target_size=(224, 224),
batch_size=4,
class_mode='binary'
)
# Let's set up the test data generator
# Uses the same parameters for consistency across training, validation, and testing
test_generator = my_generator.flow_from_directory(
f"{base_directory}/test/",
target_size=(224, 224),
batch_size=4,
class_mode='binary'
)
Found 200 images belonging to 2 classes. Found 200 images belonging to 2 classes.
import numpy as np
from keras.models import load_model
# Load the model
model = load_model("keras_model.h5", compile=False)
# Load the labels
class_names = open("labels.txt", "r").readlines()
def predict_and_evaluate_from_generator(generator, num_batches):
# To hold all predictions and true labels
all_predictions = []
all_true_labels = []
# Loop over the generator to collect predictions
for _ in range(num_batches):
images, true_labels = next(generator)
predictions = model.predict(images, verbose=0)
predicted_labels = np.argmax(predictions, axis=1)
all_predictions.extend(predicted_labels)
all_true_labels.extend(true_labels)
# Convert all predictions and true labels to numpy arrays
all_predictions = np.array(all_predictions)
all_true_labels = np.array(all_true_labels)
# Calculate accuracy and precision
accuracy = accuracy_score(all_true_labels, all_predictions)
# Note: Set the average type according to your classification (binary, micro, macro, weighted, etc.)
precision = precision_score(all_true_labels, all_predictions, average='binary') # Adjust average as necessary
# Print accuracy and precision
print(f"Accuracy: {accuracy}")
print(f"Precision: {precision}")
num_batches = 5 # Define how many batches you want to use for evaluation
print("Evaluation on validation data:")
predict_and_evaluate_from_generator(valid_generator, num_batches)
Evaluation on validation data: Accuracy: 0.7 Precision: 0.75
num_batches = 5 # Define how many batches you want to use for evaluation
print("Evaluation on test data:")
predict_and_evaluate_from_generator(test_generator, num_batches)
Evaluation on test data: Accuracy: 0.7 Precision: 1.0
Using the Teachable Machines browser tool to fine-tune a political meme dataset resulted in a validation accuracy of 0.7. The test performance decreased compared to the base configuration; the accuracy with the Teachable Machines architecture was 0.7, while the accuracy with the base configuration was 0.9550.
Increasing the number of epochs frequently causes the web page to become unresponsive, necessitating repeated reloads of the website and re-uploads of images, which complicates the entire process.
After exporting the model and loading it using Keras to validate the validation and test accuracy, the accuracy changes every time I run the code. Typically, after training and loading a model, the validation and test accuracy should not vary.
# Define the input shape and number of classes
input_shape = (150, 150, 3)
num_classes = 2 # for binary classification
# Start defining the model using the Functional API
inputs = keras.Input(shape=input_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(inputs)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(128, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(256, 3, padding='same', activation='relu')(x)
x = layers.MaxPooling2D(pool_size=2)(x)
# Global Average Pooling serves as a way to reduce spatial dimensions to a single vector per feature map
x = layers.GlobalAveragePooling2D()(x)
# Output layer for binary classification
outputs = layers.Dense(1, activation='sigmoid')(x)
# Finalize and create the model
model = keras.Model(inputs=inputs, outputs=outputs)
model.summary()
Model: "model_7"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_8 (InputLayer) [(None, 150, 150, 3)] 0
conv2d_47 (Conv2D) (None, 150, 150, 32) 896
max_pooling2d_47 (MaxPooli (None, 75, 75, 32) 0
ng2D)
conv2d_48 (Conv2D) (None, 75, 75, 64) 18496
max_pooling2d_48 (MaxPooli (None, 37, 37, 64) 0
ng2D)
conv2d_49 (Conv2D) (None, 37, 37, 128) 73856
max_pooling2d_49 (MaxPooli (None, 18, 18, 128) 0
ng2D)
conv2d_50 (Conv2D) (None, 18, 18, 128) 147584
max_pooling2d_50 (MaxPooli (None, 9, 9, 128) 0
ng2D)
conv2d_51 (Conv2D) (None, 9, 9, 256) 295168
max_pooling2d_51 (MaxPooli (None, 4, 4, 256) 0
ng2D)
global_average_pooling2d_6 (None, 256) 0
(GlobalAveragePooling2D)
dense_15 (Dense) (None, 1) 257
=================================================================
Total params: 536257 (2.05 MB)
Trainable params: 536257 (2.05 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
cb_early = keras.callbacks.EarlyStopping(
monitor="val_loss",
patience=30
)
# Let's compile the CNN model using binary cross_entropy as loss function and adam as optimizer
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy', keras.metrics.Precision()])
# Let's train and validate model on the training and validation data
history = model.fit(train_generator, validation_data = valid_generator, epochs = 30, verbose = 1, batch_size = 8, callbacks = [cb_early])
Epoch 1/30 150/150 [==============================] - 6s 27ms/step - loss: 0.6710 - accuracy: 0.6050 - precision_5: 0.6567 - val_loss: 0.4793 - val_accuracy: 0.7900 - val_precision_5: 0.9265 Epoch 2/30 150/150 [==============================] - 4s 25ms/step - loss: 0.6210 - accuracy: 0.7050 - precision_5: 0.6519 - val_loss: 0.4338 - val_accuracy: 0.9050 - val_precision_5: 0.8584 Epoch 3/30 150/150 [==============================] - 4s 26ms/step - loss: 0.4390 - accuracy: 0.8317 - precision_5: 0.7901 - val_loss: 0.5119 - val_accuracy: 0.7500 - val_precision_5: 0.6667 Epoch 4/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3521 - accuracy: 0.8533 - precision_5: 0.8313 - val_loss: 0.2643 - val_accuracy: 0.8900 - val_precision_5: 0.8197 Epoch 5/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3031 - accuracy: 0.8833 - precision_5: 0.8639 - val_loss: 0.3662 - val_accuracy: 0.8350 - val_precision_5: 0.7519 Epoch 6/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2957 - accuracy: 0.8933 - precision_5: 0.8806 - val_loss: 0.2907 - val_accuracy: 0.8550 - val_precision_5: 0.7752 Epoch 7/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2468 - accuracy: 0.9017 - precision_5: 0.8951 - val_loss: 0.2584 - val_accuracy: 0.9150 - val_precision_5: 0.9029 Epoch 8/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2746 - accuracy: 0.8950 - precision_5: 0.8990 - val_loss: 0.3227 - val_accuracy: 0.8500 - val_precision_5: 0.7734 Epoch 9/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2190 - accuracy: 0.9233 - precision_5: 0.9123 - val_loss: 0.2483 - val_accuracy: 0.8950 - val_precision_5: 0.8264 Epoch 10/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2082 - accuracy: 0.9200 - precision_5: 0.9257 - val_loss: 0.4277 - val_accuracy: 0.8100 - val_precision_5: 0.7246 Epoch 11/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1835 - accuracy: 0.9317 - precision_5: 0.9331 - val_loss: 0.2966 - val_accuracy: 0.8850 - val_precision_5: 0.8130 Epoch 12/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1805 - accuracy: 0.9233 - precision_5: 0.9291 - val_loss: 0.3917 - val_accuracy: 0.8450 - val_precision_5: 0.7634 Epoch 13/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1730 - accuracy: 0.9400 - precision_5: 0.9400 - val_loss: 0.1531 - val_accuracy: 0.9400 - val_precision_5: 0.9000 Epoch 14/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1709 - accuracy: 0.9333 - precision_5: 0.9305 - val_loss: 0.3064 - val_accuracy: 0.8500 - val_precision_5: 0.7692 Epoch 15/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1528 - accuracy: 0.9400 - precision_5: 0.9400 - val_loss: 0.1737 - val_accuracy: 0.9300 - val_precision_5: 0.8772 Epoch 16/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1652 - accuracy: 0.9300 - precision_5: 0.9358 - val_loss: 0.2502 - val_accuracy: 0.8950 - val_precision_5: 0.8264 Epoch 17/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1328 - accuracy: 0.9483 - precision_5: 0.9498 - val_loss: 0.3321 - val_accuracy: 0.8750 - val_precision_5: 0.8000 Epoch 18/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0989 - accuracy: 0.9617 - precision_5: 0.9632 - val_loss: 0.3960 - val_accuracy: 0.8900 - val_precision_5: 0.8197 Epoch 19/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0944 - accuracy: 0.9617 - precision_5: 0.9695 - val_loss: 0.3759 - val_accuracy: 0.8700 - val_precision_5: 0.7937 Epoch 20/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0900 - accuracy: 0.9650 - precision_5: 0.9697 - val_loss: 0.2035 - val_accuracy: 0.9200 - val_precision_5: 0.8818 Epoch 21/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0877 - accuracy: 0.9667 - precision_5: 0.9730 - val_loss: 0.3680 - val_accuracy: 0.8700 - val_precision_5: 0.7937 Epoch 22/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0883 - accuracy: 0.9717 - precision_5: 0.9732 - val_loss: 0.4260 - val_accuracy: 0.8550 - val_precision_5: 0.7752 Epoch 23/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0559 - accuracy: 0.9833 - precision_5: 0.9801 - val_loss: 0.4858 - val_accuracy: 0.9050 - val_precision_5: 0.8403 Epoch 24/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0802 - accuracy: 0.9633 - precision_5: 0.9572 - val_loss: 0.1835 - val_accuracy: 0.9400 - val_precision_5: 0.8929 Epoch 25/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0745 - accuracy: 0.9733 - precision_5: 0.9702 - val_loss: 0.4261 - val_accuracy: 0.8700 - val_precision_5: 0.7937 Epoch 26/30 150/150 [==============================] - 4s 25ms/step - loss: 0.0477 - accuracy: 0.9867 - precision_5: 0.9834 - val_loss: 0.6151 - val_accuracy: 0.8500 - val_precision_5: 0.7692 Epoch 27/30 150/150 [==============================] - 4s 26ms/step - loss: 0.0682 - accuracy: 0.9817 - precision_5: 0.9801 - val_loss: 0.2826 - val_accuracy: 0.9000 - val_precision_5: 0.8333 Epoch 28/30 150/150 [==============================] - 4s 24ms/step - loss: 0.2215 - accuracy: 0.9417 - precision_5: 0.9344 - val_loss: 0.1733 - val_accuracy: 0.9150 - val_precision_5: 0.8547 Epoch 29/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1016 - accuracy: 0.9633 - precision_5: 0.9664 - val_loss: 0.4181 - val_accuracy: 0.8450 - val_precision_5: 0.7634 Epoch 30/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1114 - accuracy: 0.9567 - precision_5: 0.9507 - val_loss: 0.1700 - val_accuracy: 0.9450 - val_precision_5: 0.9009
train_accuracy = history.history["accuracy"]
train_loss = history.history["loss"]
train_precision = history.history["precision_5"]
val_accuracy = history.history["val_accuracy"]
val_loss = history.history["val_loss"]
val_precision = history.history["val_precision_5"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's test the model
model.evaluate(test_generator)
50/50 [==============================] - 1s 17ms/step - loss: 0.3606 - accuracy: 0.9050 - precision_5: 0.8403
[0.36062026023864746, 0.9049999713897705, 0.8403361439704895]
Using the code provided by ChatGPT to build, compile, and fine-tune a political meme dataset resulted in a validation accuracy of 0.9450. The test performance was almost similar to the base configuration; the accuracy with the ChatGPT architecture was 0.91, while the accuracy with the base configuration was 0.9550.
Using ChatGPT to train and validate a political images dataset resulted in code that had several issues, particularly with the incorrect number of neurons in the dense layer, leading to repeated graph execution errors. Despite multiple attempts to resolve the issue through ChatGPT, the problem persisted, and I ultimately had to fix it myself.
After resolving all the issues for training, the validation accuracy fluctuated randomly. When asked, ChatGPT produced different versions of code that did not make any sense. Furthermore, the explanations it provided for the fluctuations were not logical.
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
# Define the model architecture
model_gemini = Sequential()
model_gemini.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(150, 150, 3))) # Update input shape
model_gemini.add(MaxPooling2D((2, 2)))
model_gemini.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model_gemini.add(MaxPooling2D((2, 2)))
# model.add(Flatten())
model_gemini.add(layers.GlobalAveragePooling2D())
model_gemini.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model_gemini.add(Dense(1, activation='sigmoid')) # Output layer for binary classification
# Compile the model
model_gemini.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy', keras.metrics.Precision()])
# Train the model
history_gemini = model_gemini.fit(train_generator, epochs=30, validation_data=valid_generator)
Epoch 1/30 150/150 [==============================] - 5s 27ms/step - loss: 0.6630 - accuracy: 0.5733 - precision_8: 0.6134 - val_loss: 0.5693 - val_accuracy: 0.7100 - val_precision_8: 0.6750 Epoch 2/30 150/150 [==============================] - 4s 26ms/step - loss: 0.5577 - accuracy: 0.7150 - precision_8: 0.7416 - val_loss: 0.4817 - val_accuracy: 0.8150 - val_precision_8: 0.7890 Epoch 3/30 150/150 [==============================] - 4s 26ms/step - loss: 0.4771 - accuracy: 0.7900 - precision_8: 0.7939 - val_loss: 0.3996 - val_accuracy: 0.8400 - val_precision_8: 0.7881 Epoch 4/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3992 - accuracy: 0.8250 - precision_8: 0.8095 - val_loss: 0.3371 - val_accuracy: 0.8700 - val_precision_8: 0.7984 Epoch 5/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3558 - accuracy: 0.8483 - precision_8: 0.8276 - val_loss: 0.4387 - val_accuracy: 0.7750 - val_precision_8: 0.6897 Epoch 6/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3103 - accuracy: 0.8817 - precision_8: 0.8658 - val_loss: 0.4433 - val_accuracy: 0.7700 - val_precision_8: 0.6849 Epoch 7/30 150/150 [==============================] - 4s 25ms/step - loss: 0.3263 - accuracy: 0.8667 - precision_8: 0.8526 - val_loss: 0.5119 - val_accuracy: 0.7300 - val_precision_8: 0.6494 Epoch 8/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2756 - accuracy: 0.9050 - precision_8: 0.8932 - val_loss: 0.2183 - val_accuracy: 0.9200 - val_precision_8: 0.8684 Epoch 9/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2642 - accuracy: 0.8967 - precision_8: 0.8839 - val_loss: 0.3593 - val_accuracy: 0.8150 - val_precision_8: 0.7299 Epoch 10/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2611 - accuracy: 0.9050 - precision_8: 0.8958 - val_loss: 0.4499 - val_accuracy: 0.7800 - val_precision_8: 0.6944 Epoch 11/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2403 - accuracy: 0.9117 - precision_8: 0.9049 - val_loss: 0.2149 - val_accuracy: 0.9200 - val_precision_8: 0.8621 Epoch 12/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2361 - accuracy: 0.9100 - precision_8: 0.9100 - val_loss: 0.3915 - val_accuracy: 0.8300 - val_precision_8: 0.7463 Epoch 13/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2278 - accuracy: 0.9100 - precision_8: 0.9046 - val_loss: 0.2466 - val_accuracy: 0.9100 - val_precision_8: 0.8475 Epoch 14/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2473 - accuracy: 0.9117 - precision_8: 0.9158 - val_loss: 0.4923 - val_accuracy: 0.7800 - val_precision_8: 0.6944 Epoch 15/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2326 - accuracy: 0.9050 - precision_8: 0.9037 - val_loss: 0.2016 - val_accuracy: 0.9250 - val_precision_8: 0.8696 Epoch 16/30 150/150 [==============================] - 4s 26ms/step - loss: 0.2178 - accuracy: 0.9217 - precision_8: 0.9148 - val_loss: 0.2152 - val_accuracy: 0.9200 - val_precision_8: 0.8621 Epoch 17/30 150/150 [==============================] - 4s 27ms/step - loss: 0.2037 - accuracy: 0.9183 - precision_8: 0.9226 - val_loss: 0.2887 - val_accuracy: 0.8800 - val_precision_8: 0.8065 Epoch 18/30 150/150 [==============================] - 4s 27ms/step - loss: 0.2042 - accuracy: 0.9117 - precision_8: 0.9076 - val_loss: 0.2837 - val_accuracy: 0.8600 - val_precision_8: 0.7812 Epoch 19/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1958 - accuracy: 0.9350 - precision_8: 0.9394 - val_loss: 0.3034 - val_accuracy: 0.8600 - val_precision_8: 0.7812 Epoch 20/30 150/150 [==============================] - 4s 27ms/step - loss: 0.1888 - accuracy: 0.9267 - precision_8: 0.9267 - val_loss: 0.2670 - val_accuracy: 0.8900 - val_precision_8: 0.8197 Epoch 21/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1981 - accuracy: 0.9267 - precision_8: 0.9324 - val_loss: 0.1672 - val_accuracy: 0.9300 - val_precision_8: 0.8772 Epoch 22/30 150/150 [==============================] - 4s 25ms/step - loss: 0.2104 - accuracy: 0.9100 - precision_8: 0.9184 - val_loss: 0.2236 - val_accuracy: 0.9150 - val_precision_8: 0.8547 Epoch 23/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1802 - accuracy: 0.9317 - precision_8: 0.9390 - val_loss: 0.4033 - val_accuracy: 0.8450 - val_precision_8: 0.7634 Epoch 24/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1950 - accuracy: 0.9267 - precision_8: 0.9267 - val_loss: 0.2226 - val_accuracy: 0.9150 - val_precision_8: 0.8547 Epoch 25/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1764 - accuracy: 0.9367 - precision_8: 0.9549 - val_loss: 0.2855 - val_accuracy: 0.8850 - val_precision_8: 0.8130 Epoch 26/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1805 - accuracy: 0.9367 - precision_8: 0.9486 - val_loss: 0.1975 - val_accuracy: 0.9250 - val_precision_8: 0.8696 Epoch 27/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1731 - accuracy: 0.9367 - precision_8: 0.9456 - val_loss: 0.2244 - val_accuracy: 0.9100 - val_precision_8: 0.8475 Epoch 28/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1756 - accuracy: 0.9333 - precision_8: 0.9452 - val_loss: 0.4865 - val_accuracy: 0.8150 - val_precision_8: 0.7299 Epoch 29/30 150/150 [==============================] - 4s 26ms/step - loss: 0.1713 - accuracy: 0.9317 - precision_8: 0.9390 - val_loss: 0.4026 - val_accuracy: 0.8450 - val_precision_8: 0.7634 Epoch 30/30 150/150 [==============================] - 4s 25ms/step - loss: 0.1789 - accuracy: 0.9350 - precision_8: 0.9516 - val_loss: 0.2161 - val_accuracy: 0.9300 - val_precision_8: 0.8772
train_accuracy = history_gemini.history["accuracy"]
train_loss = history_gemini.history["loss"]
train_precision = history_gemini.history["precision_8"]
val_accuracy = history_gemini.history["val_accuracy"]
val_loss = history_gemini.history["val_loss"]
val_precision = history_gemini.history["val_precision_8"]
epochs = range(1, len(train_accuracy) + 1)
plt.plot(epochs, train_accuracy, "bo", label="Training accuracy")
plt.title("Training Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, train_loss, "bo", label="Training loss")
plt.title("Training Loss")
plt.legend()
plt.show()
plt.plot(epochs, train_precision, "bo", label="Training precision")
plt.title("Training Precision")
plt.legend()
plt.show()
plt.plot(epochs, val_accuracy, "bo", label="Validation accuracy")
plt.title("Validation Accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, val_loss, "bo", label="Validation loss")
plt.title("Validation Loss")
plt.legend()
plt.show()
plt.plot(epochs, val_precision, "bo", label="Validation precision")
plt.title("Validation Precision")
plt.legend()
plt.show()
# Let's test the model
model_gemini.evaluate(test_generator)
50/50 [==============================] - 1s 16ms/step - loss: 0.3590 - accuracy: 0.8750 - precision_8: 0.8000
[0.3590352535247803, 0.875, 0.800000011920929]
Using the code provided by Gemini to build, compile, and fine-tune a political meme dataset resulted in a validation accuracy of 0.93. The test performance was almost similar to the base configuration; the accuracy with the Gemini architecture was 0.88, while the accuracy with the base configuration was 0.9550.
Using Gemini to train and validate a political images dataset resulted in code that had several issues, particularly with the incorrect number of neurons in the dense layer, leading to repeated graph execution errors. Despite multiple attempts to resolve the issue through Gemini, the problem persisted, and I ultimately had to fix it myself.
After resolving all the issues for training, the validation accuracy fluctuated randomly. When asked, Gemini produced different versions of code that did not make any sense. Furthermore, the explanations it provided for the fluctuations were not logical.
| Tool | Validation Accuracy | Test Accuracy |
|---|---|---|
| Teachable Machines | 0.7 | 0.7 |
| ChatGPT | 0.9450 | 0.91 |
| Gemini | 0.93 | 0.88 |